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(54) Mirroring agent accessible to remote host computers, and accessing remote data-storage 
devices, via a communications medium 



(57) A hardware-based mirroring agent (1602) that 
provides a LUN-based I/O interface to remote host com- 
puters (1606), including mirrored LUNs, implemented 
via remote data storage devices (1 608, 1610). The hard- 



ware-based mirroring agent is similar to a disk array, but 
manages and provides to host computers an interface 
to remote data storage devices, rather than to internal 
data storage devices, as in the case of disk arrays. 
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Description 

TECHNICAL FIELD 

[0001] The present invention is related to replication 
and concurrent updating of multiple physical storage de- 
vices to provide fail-over data redundancy, and, in par- 
ticular, to a hardware-based mirroring agent, intercon- 
nected with remote host computers and remote data- 
storage devices via a communications medium, that co- 
ordinates and manages mirroring of remote data-stor- 
age devices and that provides to host computers a sim- 
ple interface to mirrored physical data-storage devices. 

BACKGROUND OF THE INVENTION 

[0002] The present invention relates to concurrently 
synchronized, redundant storage of data on multiple 
mass storage devices. The present invention is de- 
scribed and illustrated with reference to an embodiment 
similar, in many respects, to a disk array that services I/ 
O requests from a number of remote computers. There- 
fore, an overview of mass storage devices, disk arrays, 
and disk mirroring is provided, below. 
[0003] Figure 1 illustrates data storage within a platter 
of a hard disk drive. The platter is a thin disk, coated 
with a magnetic medium, such as iron oxide. Data can 
be stored in tiny areas of the surface of the platter having 
induced, stable magnetic fields. The surface of the disk 
platter 102 is divided into concentric rings, or tracks, 
such as tracks 1 04-1 05 in Figure 1 . Current disk platters 
contain many thousands of tracks. Each track is divided 
into radial segments, or sectors, such as sector 1 06 of 
track 1 04 in Figure 1 . Sectors each normally comprise 
a fixed number of bytes, normally 256, 512, 1024, or 
2048 bytes. Data is normally retrieved from, and stored 
to, a hard disk drive in units of sectors. Once a sector is 
read from a disk and stored into computer memory, a 
program may access individual bytes and bits within the 
sector by accessing the random memory in which the 
sector is stored. Thus, the physical location of data on 
a disk platter can be described by a starting location and 
an ending location, each location specified as a track/ 
sector/byte triple. Normally, a hard disk drive contains a 
number of platters aligned in parallel along a spindle 
passing through the center of each platter. Typically, the 
track and sectors of the platter can be thought of as 
aligned to form cylinders spanning the platters. In such 
hard disk drives, the physical address of a byte of data 
may also be described by a track/sector/byte triplet, 
where the byte within an aligned group of sectors com- 
posing a section of a cylinder are consecutively ordered. 
[0004] Figure 2 is a block diagram of a standard disk 
drive. The disk drive 201 receives input/output ("I/O") 
requests from remote computers via a communications 
medium 202 such as a computer bus, fibre channel, or 
other such electronic communications medium. For 
many types of storage devices, including the disk drive 



201 illustrated in Figure 2, the vast majority of I/O re- 
quests are either READ or WRITE requests. A READ 
request requests that the storage device return to the 
requesting remote computer some requested amount of 

5 electronic data stored within the storage device. A 
WRITE request requests that the storage device store 
electronic data furnished by the remote computer within 
the storage device. Thus, as a result of a READ opera- 
tion carried out by the storage device, data is returned 

10 via communications medium 202 to a remote computer, 
and as a result of a WRITE operation, data is received 
from a remote computer by the storage device via com- 
munications medium 202 and stored within the storage 
device. 

15 [0005] The disk drive storage device illustrated in Fig- 
ure 2 includes controller hardware and logic 203 includ- 
ing electronic memory, one or more processors or 
processing circuits, and controller firmware, and also in- 
cludes a number of disk platters 204 coated with a mag- 
20 netic medium for storing electronic data. The disk drive 
contains many other components not shown in Figure 
2, including read/write heads, a high-speed electronic 
motor, a drive shaft, and other electronic, mechanical, 
and electromechanical components. The memory with- 
25 in the disk drive includes a request/reply buffer 205 
which stores I/O requests received from remote com- 
puters and an I/O queue 206 that stores internal I/O 
commands corresponding to the I/O requests stored 
within the request/reply buffer 205. Communication be- 
so tween remote computers and the disk drive, translation 
of I/O requests into internal I/O commands, and man- 
agement of the I/O queue, among other things, are car- 
ried out by the disk drive I/O controller as specified by 
disk drive I/O controller firmware 207. Translation of in- 
35 ternal I/O commands into electromechanical disk oper- 
ations in which data is stored onto, or retrieved from, the 
disk platters 204 is carried out by the disk drive I/O con- 
troller as specified by disk media read/write manage- 
ment firmware 208. Thus, the disk drive I/O control 
40 firmware 207 and the disk media read/write manage- 
ment firmware 208, along with the processors and mem- 
ory that enable execution of the firmware, compose the 
disk drive controller. 

[0006] Individual disk drives, such as the disk drive 
45 illustrated in Figure 2, are normally connected to, and 
used by, a single remote computer, although it has been 
common to provide dual-ported disk drives for concur- 
rent use by two computers and multi-host-accessible 
disk drives that can be accessed by numerous remote 
50 computers via a communications medium such as a fi- 
bre channel. However, the amount of electronic data 
that can be stored in a single disk drive is limited. In order 
to provide much larger-capacity electronic data-storage 
devices that can be efficiently accessed by numerous 
55 remote computers, disk manufacturers commonly com- 
bine many different individual disk drives, such as the 
disk drive illustrated in Figure 2, into a disk array device, 
increasing both the storage capacity as well as increas- 
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ing the capacity for parallel I/O request servicing by con- 
current operation of the multiple disk drives contained 
within the disk array. 

[0007] Figure 3 is a simple block diagram of a disk 
array. The disk array 302 includes a number of disk drive 5 
devices 303, 304, and 305. In Figure 3, for simplicity of 
illustration, only three individual disk drives are shown 
within the disk array, but disk arrays may contain many 
tens or hundreds of individual disk drives. A disk array 
contains a disk array controller 306 and cache memory 10 
307. Generally, data retrieved from disk drives in re- 
sponse to READ requests may be stored within the 
cache memory 307 so that subsequent requests for the 
same data can be more quickly satisfied by reading the 
data from the quickly accessible cache memory rather 15 
than from the much slower electromechanical disk 
drives. Various elaborate mechanisms are employed to 
maintain, within the cache memory 307, data that has 
the greatest chance of being subsequently re-requested 
within a reasonable amount of time. The disk array con- 20 
troller 306 may also elect to store data received from 
remote computers via WRITE requests in cache mem- 
ory 307 in the event that the data may be subsequently 
requested via READ requests or in order to defer slower 
writing of the data to physical storage medium. 25 
[0008] Electronic data is stored within a disk array at 
specific addressable locations. Because a disk array 
may contain many different individual disk drives, the 
address space represented by a disk array is immense, 
generally many thousands of gigabytes. The overall ad- 30 
dress space is normally partitioned among a number of 
abstract data storage resources called logical units 
("LUNs"). A LUN includes a defined amount of electronic 
data storage space, mapped to the data storage space 
of one or more disk drives within the disk array, and may 35 
be associated with various logical parameters including 
access privileges, backup frequencies, and mirror coor- 
dination with one or more LUNs. LUNs may also be 
based on random access memory ("RAM"), mass stor- 
age devices other than hard disks, or combinations of *o 
memory, hard disks, and/or other types of mass storage 
devices. Remote computers generally access data with- 
in a disk array through one of the many abstract LUNs 
308-315 provided by the disk array via internal disk 
drives 303-305 and the disk array controller 306. Thus, 45 
a remote computer may specify a particular unit quantity 
of data, such as a byte, word, or block, using a bus com- 
munications media address corresponding to a disk ar- 
ray, a LUN specifier, normally a 64-bit integer, and a 
32-bit, 64-bit, or 128-bit data address that specifies a so 
LUN, and a data address within the logical data address 
partition allocated to the LUN. The disk array controller 
translates such a data specification into an indication of 
a particular disk drive within the disk array and a logical 
data address within the disk drive. A disk drive controller 55 
within the disk drive finally translates the logical address 
to a physical medium address. Normally, electronic data 
is read and written as one or more blocks of contiguous 



32-bit or 64-bit computer words, the exact details of the 
granularity of access depending on the hardware and 
firmware capabilities within the disk array and individual 
disk drives as well as the operating system of the remote 
computers generating I/O requests and characteristics 
of the communication medium interconnecting the disk 
array with the remote computers. 
[0009] While the disk array, as described above, pro- 
vides data storage within, and addressed relative to, 
LUNs, high-level application programs ("APPs") execut- 
ing on host computers access data stored within LUNs 
via a number of higher-level abstractions. Figure 3 illus- 
trates the hierarchical data abstraction levels within a 
host computer/disk array system. Each block in Figure 
4 represents a separate program, program/hardware, or 
hardware component within the host computer/disk ar- 
ray system. As discussed above, the disk array 402 ac- 
cesses data stored within internal disks via internal 
physical addresses that each contain indications of a 
disk, a track within a disk, a sector within the track, and 
a byte within the sector. However, as discussed above, 
the disk array provides data access and storage to vir- 
tual storage spaces, called LUNs, each LUN having 
some fixed number of addressable units, such as bytes. 
The two abstractions 404 and 406 in Figure 4 are linked 
to operating system components that execute within the 
operating system of a host computer interconnected 
with a disk array. The first component is a volume man- 
ager 404. This component interacts with a disk array via 
a communications medium, accessing and storing data 
relative to the'LUN abstraction provided by the disk ar- 
ray. The volume manager 404 presents a different inter- 
face to components above the volume manager in the 
abstraction hierarchy. The volume manager provides 
volumes which have volume names and which contain 
a linear address space of bytes, words, or some other 
convenient addressable entity. The volume manager 
may map a volume onto one or more LUNs, translating 
volume-relative addresses received from higher-level 
components into LUN-based data addresses that the 
volume manager then passes to the disk array. In addi- 
tion, the volume manager can increase the size of a log- 
ical volume using an arbitrary LUN, which can quickly 
change the physical location ofthe entirety of a logical 
object. 

[0010] The next highest component shown in Figure 
4 is the operating system's file manager 406. The file 
manager provides a logical object interface to the high- 
est-level component, an executing APP 408. Most log- 
ical objects currently provided by file managers and 
used by APPs are called "files." Files are arbitrarily 
sized, consecutive sequences of data bytes, described 
by file names, that are stored on a mass storage device 
and read from, and written to, via operating-system-pro- 
vided I/O commands. A file manager provides a hierar- 
chical, multi-component file name space to allow an 
APP or user to organize files within hierarchical directo- 
ries. The file manager translates a file name, including 
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the directory and subdirectory prefixes within the file 
name, to a range of consecutive addressable entities, 
such as bytes, within a volume. An APP 408, or a human 
user interacting with the APP, stores data to, and ac- 
cesses data from, a mass storage device, such as a disk 5 
array, in terms of named logical objects. 
[0011] In many computer applications and systems 
that need to reliably store and retrieve data from a mass 
storage device, such as a disk array, a primary data ob- 
ject, such as a file or database, is normally backed up 10 
to backup copies of the primary data object on physically 
discrete mass storage devices or media so that if, during 
operation of the application or system, the primary data 
object becomes corrupted, inaccessible, or is overwrit- 
ten or deleted, the primary data object can be restored is 
by copying a backup copy of the primary data object 
from the mass storage device. Many different tech- 
niques and methodologies for maintaining backup cop- 
ies have been developed. In one well-known technique, 
a primary data object is mirrored. 20 
[0012] Figure 5 illustrates object-level mirroring. In 
Figure 5, a primary data object "0 3 M 501 is stored on 
LUN A 502. The mirror object, or backup copy, H 0 3 u 503 
is stored on LUN B 504. The arrows in Figure 5, such 
as arrow 505, indicate I/O WRITE requests directed to 25 
various objects stored on a LUN. I/O WRITE requests 
directed to object "0 3 " are represented by arrow 506. 
When object-level mirroring is enabled, the disk array 
controller providing LUNs A and B automatically gener- 
ates a second I/O WRITE request from each I/O WRITE 30 
request 506 directed to LUN A, and directs the second 
generated I/O WRITE request via path 507, switch "S/ 
508, and path 509 to the mirror object "0 3 M 503 stored 
on LUN B 504. In Figure 5, enablement of mirroring is 
logically represented by switch "S^ " 508 being on. Thus, 35 
when object-level mirroring is enabled, any I/O WRITE 
request, or any other type of I/O operation that changes 
the representation of object "03" 501 on LUN A, is au- 
tomatically mirrored by the disk array controller to iden- 
tically change the mirror object ,, 0 3 " 503. Mirroring can *o 
be disabled, represented in Figure 5 by switch "S^' 508 
being in an off position. In that case, changes to the pri- 
mary data object ,, 0 3 H 501 are no longer automatically 
reflected in the mirror object "0 3 u 503. Thus, at the point 
that mirroring is disabled, the stored representation, or 45 
state, of the primary data object "0 3 " 501 may diverge 
from the stored representation, or state, of the mirror 
object "0 3 " 503. Once the primary and mirror copies of 
an object have diverged, the two copies can be brought 
back to identical representations, or states, by a resync so 
operation represented in Figure 5 by switch M S 2 M 51 0 be- 
ing in an on position. In the normal mirroring operation, 
switch "S 2 n 51 0 is in the off position. During the resync 
operation, any I/O operations that occurred after mirror- 
ing was disabled are logically issued by the disk array 55 
controller to the mirror copy of the object via path 511, 
switch "S 2 ," and pass 509. During resync, switch "Sj" is 
in the off position. Once the resync operation is com- 



plete, logical switch "S 2 M is disabled and logical switch 
"S-i" 508 can be turned on in order to reenable mirroring 
so that subsequent I/O WRITE requests or other I/O op- 
erations that change the storage state of primary data 
object "0 3 ," are automatically reflected to the mirror ob- 
ject u 0 3 " 503. Another operation generally provided as 
part of mirroring is an instant snapshot feature. Upon 
receiving a request for an instant snapshot, the disk ar- 
ray controller copies the contents of either LUN A or LUN 
B to a specified LUN, providing an incremental backup 
of the contents of the mirrored LUNS at a specified point 
in time. 

[0013] Figure 6 illustrates a hypothetical computing 
environment including host computers and data-storage 
devices interconnected by a communications medium. 
The communications environment illustrated in Figure 
6 will be reused repeatedly in subsequent discussions 
to illustrate current mirroring techniques and, later, to 
illustrate one embodiment of the present invention. Sub- 
sequent illustrations based on Figure 6 will employ nu- 
merical labels identical to numerical labels employed in 
Figure 6, for the sake of clarity. In Figure 6, a first, logi- 
cally circular communications medium 602, such as a 
fibre channel arbitrated loop, is interconnected through 
a router or bridge device 604 to a second, logically cir- 
cular communications medium 606. Host computers 
608 and 609, disk arrays 610-612, and storage devices 
61 4-61 6 are directly connected to communications me- 
dium 602, while host computers 618-619, disk arrays 
620-621 , and storage devices 622-627 are directly con- 
nected to logically circular communications medium 
606. In this environment, LUN 2628 of disk array 612 is 
a mirror copy of data storage device 616, and disk stor- 
age device 624 is a mirror copy of disk storage device 
623, as indicated in Figure 6 by dashed arrows 629 and 
630, respectively. Of course, an administrator of the 
computing environment illustrated in Figure 6 can 
choose to mirror any number of data storage devices 
illustrated in Figure 6 in any number of different ways. 
The mirroring illustrated in Figure 6 is a hypothetical ex- 
ample used to illustrate currently available mirroring 
techniques and an embodiment of the present invention. 
[0014] Currently, three general types of mirroring 
techniques are employed to provide fail-over data re- 
dundancy within computing environments, such as the 
computing environment illustrated in Figure 6. Figure 7 
illustrates a host-computer-based mirroring technique. 
In Figure 7, the host computer 702 manages mirroring 
of LUN 2 704 of disk array 706 and a storage device 708 
interconnected with each other and with host computer 
702 via a communications medium 710. This technique 
might be employed, for example, by host computer 608 
to mirror LUN 2 628 of disk array 612 and data storage 
device 61 6 in Figure 6. In this host-computer-based mir- 
roring technique, the host computer maintains one or 
more internal tables 71 2 that store a representation of 
the fact that LUN 2 704 of disk array 606 is a mirror copy 
of data storage device 708. Often, this table or tables 
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will be maintained within the volume manager of the host 
computer, although mirroring can be managed at vari- 
ous levels within a host computer, including various lev- 
els within the host computer's operating system. Gen- 
erally, when a program on the host computer issues a I/ 5 
0 WRITE request directed to one of the two mirrored 
data-storage devices 704 and 708, the operating sys- 
tem of host computer 702 automatically generates a 
second I/O WRITE request directed to the other of the 
two mirrored data-storage devices 704 and 708. Read 10 
operations and other operations that do not effect the 
data storage state of the data-storage devices need not 
be duplicated. 

[0015] Unfortunately, the host-computer-based mir- 
roring technique illustrated in Figure 7 has a number of 15 
significant disadvantages. First, the implementation of 
this mirroring technique is highly operating-system spe- 
cific, and thus may need to be separately implemented 
for each different type of host computer within a com- 
puting environment. Moreover, either the mirroring is on- 20 
ly available to host computer 702, and not to other host 
computers interconnected with communications medi- 
um 71 0, or I/O WRITE requests directed to mirrored da- 
ta-storage devices 704 and 708 must be coordinated 
between host computers to very complex and error- 25 
prone protocols and procedures. Such coordination can 
lead to significant data storage inefficiencies, for exam- 
ple requiring host computer 702 to act in an intermediary 
role for I/O requests directed to mirror data-storage de- 
vices 704 and 708 by other host computers. Another dis- 30 
advantage is that host computer represents a single 
point of failure for the mirroring of the two data storage 
devices. 

[0016] Figure 8 illustrates disk-array-based mirroring 
techniques. In Figure 8, a table 802 storing indications 35 
of mirroring relationships between data-storage devic- 
es, such as data-storage devices 804-806 contained 
within the disk array 800, is maintained within the RAM 
memory 808 within the disk array 800, and is employed 
by the disk-array controller 810 to provide mirroring of *o 
data-storage devices contained within the disk array. 
For example, a computing environment administrator 
may issue commands to the disk-array controller81 0 to 
mirror data-storage devices 804 and 805. In response, 
the disk-array controller 81 0 stores an indication of this 45 
mirroring relationship in table 802 and issues appropri- 
ate I/O commands to bring the data state of data-storage 
devices 804 and 805 into correspondence with one an- 
other. Then, upon receipt of an I/O WRITE request di- 
rected to, for example, data storage device 804, the so 
disk-array controller 810 accesses table 802, deter- 
mines that data storage device 804 is mirrored with data 
storage device 805, and issues equivalent I/O WRITE 
requests to both data-storage devices 804 and 805. The 
mirroring operations are thus transparent to host com- 55 
puters. 

[0017] Unfortunately, like the host-based mirroring 
technique illustrated in Figure 7, the disk-array-based 



mirroring technique illustrated in Figure 8 suffers from 
significant disadvantages. A core disadvantage is that 
only data-storage devices within disk array 800 can be 
mirrored. Thus, for example, in the hypothetical comput- 
ing environment shown in Figure 6, the disk-array- 
based mirroring techniques does not provide a way to 
mirror LUN 2 628 of disk array 612 and data storage 
device 616. The second disadvantage is that, analo- 
gously to the host-based mirroring technique illustrated 
in Figure 7, the disk-array-based mirroring technique is 
quite specific to the internal disk-array controller imple- 
mentation. Different disk-arrays may require significant- 
ly different implementations of the mirroring technique. 
A third distinct disadvantage is that the disk-array con- 
troller 81 0 and many other components of the disk-array 
800 may become single points of failure that upon failure 
may prevent access to both data-storage devices of a 
mirrored data-storage-device pair. 
[0018] Figure 9 illustrates a specialized-hardware- 
based technique for data-storage-device mirroring. Fig- 
ure 9 illustrates the computing environment illustrated 
in Figure 6 with the addition of specialized mirroring 
hardware. In Figure 9, two mirroring devices 902 and 
904 have been directly connected to logically circular 
communications media 602 and 606, respectively. Stor- 
age device 61 6 and disk array 61 2, formerly directly con- 
nected to communications medium 602, have been re- 
connected directly to mirroring device 902, and storage 
devices 623 and 624, formerly directly connected to 
communications medium 606, have been reconnected 
to mirroring device 904. Mirroring devices 902 and 904 
present interfaces similar to the interface presented by 
a disk array, and contain controllers and other compo- 
nents, similar to those contained in a disk array, that 
service externally connected data-storage devices, 
such as data-storage devices 616, 623, 624, and disk 
array 612, rather than internal data-storage devices, as 
in disk arrays. Mirroring devices 902 and 904 detect re- 
ceived I/O WRITE requests directed to mirrored devices 
and generate appropriate additional I/O WRITE re- 
quests in order to maintain identical data states within 
mirrored data-storage devices connected to the mirror- 
ing devices 902 and 904. 

[0019] Like the two previously discussed mirroring 
techniques, the specialized-hardware mirroring tech- 
nique illustrated in Figure 9 suffers from significant dis- 
advantages. One disadvantage is that the number of da- 
ta-storage devices that can be mirrored by a mirroring 
device, such as mirroring device 902, may be signifi- 
cantly constrained by the number and types of external 
connections provided by the mirroring device. A second 
disadvantage is that data-storage devices must be 
physically disconnected from a communications medi- 
um and reconnected to the device. Physical removal 
may then need to be carefully coordinated with complex 
reconfigurations of host computers and other devices 
connected to the original communications medium. As 
with the previously described disk-array-based mirror- 
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ing technique, the mirroring device, such as mirroring 
device 902, may become a significant single point of fail- 
ure with respect to the mirrored data-storage devices 
that the mirroring device controls. 

[0020] Thus, as discussed above, while mirroring of 5 
data-storage devices provides convenient and needed 
data redundancy and enables rapid fail-over in the event 
of failure of a data storage device, many currently avail- 
able mirroring techniques have significant disadvantag- 
es. For that reason, designer, manufacturers, and users 10 
of data-storage devices and host computers within com- 
puting environments have recognized the need for a 
convenient and robust method and system for effecting 
data-storage-device mirroring. 

15 

SUMMARY OF THE INVENTION 

[0021] One embodiment of the present invention is a 
hardware-based mirroring agent that is interconnected, 
via a communications medium or media, to one or more 20 
remote host computers and to multiple remote data- 
storage devices. The hardware-based mirroring agent 
contains a RAM memory, controller, and controller rou- 
tines and other firmware and software similar to the 
RAM memory, controller, and controller routines and 25 
other firmware and software contained in a disk array, 
and provides a data storage interface to the one or more 
host computers similar to that provided by a disk array. 
However, unlike disk arrays and the specialized-hard- 
ware-based mirroring devices discussed above, the 30 
hardware-based mirroring agent that represents one 
embodiment of the present invention neither contains 
internal data-storage devices accessible to remote host 
computers nor provides external connections for direct 
physical connection of data-storage devices to the spe- 35 
cialized-hardware-based mirroring device. Instead, the 
hardware-based mirroring agent accesses remote data- 
storage devices via the communications medium, initial- 
izes and maintains mirror relationships between remote 
data-storage devices, and provides an interface to the *o 
mirrored remote data-storage devices to one or more 
host computers via the communications medium. The 
interface provided by the hardware-based mirroring 
agent is one or more virtual LUNs, a virtual LUN inter- 
face equivalent to a LUN interface provided by a disk 45 
array. A virtual LUN differs from a LUN in that a virtual 
LUN provided by a hardware-based mirroring agent 
stores data on, an retrieves data from, data storage de- 
vices remote from the hardware-based mirroring agent, 
whereas a LUN provided by a disk array stores data on, 50 
an retrieves data from, data storage devices included 
within the disk array. Virtual LUNs are accessed by host 
computers via a communications medium address as- 
sociated with a hardware-based mirroring agent, just as 
LUNs are accessed by host computers via a communi- 55 
cations medium address associated with a disk array or 
other type of data storage device. The hardware-based 
mirroring agent that represents one embodiment of the 



present invention is therefore essentially a diskless disk 
array that supports mirroring of remote disks, or other 
types of data-storage devices, although a hardware- 
based mirroring agent that includes an internal disk 
drive for storing mirroring-agent-internal data unavaila- 
ble via the disk-array-like interface provided to host 
computers would also fall within the scope of the present 
invention. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0022] Figure 1 illustrates data storage within a platter 
of a hard disk drive. 

[0023] Figure 2 is a block diagram of a standard disk 
drive. 

[0024] Figure 3 is a simple block diagram of a disk 
array. 

[0025] Each block in Figure 4 represents a separate 
program, program/hardware, or hardware component 
within the host computer/disk array system. 
[0026] Figure 5 illustrates object-level mirroring. 
[0027] Figure 6 illustrates a hypothetical computing 
environment including host computers and data-storage 
devices interconnected by a communications medium. 
[0028] Figure 7 illustrates a host-computer-based 
mirroring technique. 

[0029] Figure 8 illustrates disk-array-based mirroring 
techniques. 

[0030] Figure 9 illustrates a specialized-hardware- 
based technique for data-storage-device mirroring. 
[0031] Figure 10 shows the computing environment 
illustrated in Figure 6 with the addition of a hardware- 
based mirroring agent that represents one embodiment 
of the present invention. 

[0032] Figures 11-14 illustrate execution of I/O 
WRITE requests directed to a LUN provided via a disk- 
array-like interface by a mirroring agent. 
[0033] Figure 15 shows the computing environment 
illustrated in Figure 6 following addition of three mirror- 
ing agents 1002, 1502, and 1504. 
[0034] Figure 16 illustrates the logical components of 
a mirrored data storage device environment provided to 
a host computer via an intermediary hardware-based 
mirroring agent that represents one embodiment of the 
present invention. 

DETAILED DESCRIPTION OF THE INVENTION 

[0035] One embodiment of the present invention is a 
hardware-based mirroring agent interconnected 
through one or more communications media, to remote 
host computers and remote data-storage devices. The 
hardware-based mirroring agent presents to host com- 
puters an interface similar to that presented by disk ar- 
rays, namely one or more LUNs accessible via the com- 
munications medium address of the mirroring agent. 
The mirroring agent implements this interface by direct- 
ing I/O requests and commands to remote disk storage 
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devices also accessible via one or more communica- 
tions media. The mirroring agent is expressly designed 
to initialize and maintain mirror relationships between 
remote data-storage devices on behalf of the host com- 
puters, under control of a computing environment ad- 
ministrator, network administrator, or other configuration 
and management personnel. Alternatively, the mirroring 
agent may be automatically configured and managed 
via automated configuration and management tools im- 
plemented on a host computer or other computing de- 
vice interconnected to the mirroring agent via a commu- 
nications medium. The mirroring agent is thus similar to 
a disk array or to a specialized-hardware mirroring de- 
vice described above, but without internal data-storage 
devices accessible externally to host computers and 
without data-storage devices directly coupled to the mir- 
roring agent and accessible externally to host comput- 
ers via a communications medium. 
[0036] Figure 10 shows the computing environment 
illustrated in Figure 6 with the addition of a hardware- 
based mirroring agent that represents one embodiment 
of the present invention. The mirroring agent 1002 is di- 
rectly connected to the communications medium 602, 
but also be directly connected to communications me- 
dium 606. Connected to either communications medi- 
um, mirroring agent 1 002 can initialize and maintain the 
mirroring relationships illustrated in Figure 6 for both 
communications medium 602 and communications me- 
dium 606. 

[0037] Figures 11-14 illustrate execution of I/O 
WRITE requests directed to a LUN provided via a disk- 
array-like interface by a mirroring agent. In Figure 11, 
host computer 608 directs an I/O WRITE request to a 
LUN provided by the mirroring agent 1002, as indicated 
by arrow 1 1 02. The LUN provided by mirroring agent 
1002 is essentially a virtual LUN, with the data stored 
on mirrored data-storage devices 628 and 616, as pre- 
viously described with reference to Figure 6. A virtual 
LUN appears to an accessing remote host computer as 
a LUN residing at the communications medium address 
of the mirroring agent, but the data stored in the virtual 
LUN is physically stored within remote data storage de- 
vices accessed by the mirroring agent. Thus, the mirror- 
ing agent provides a virtual LUN via access to remote 
data storage devices. Upon receiving the I/O WRITE re- 
quest from host computer 608, the mirroring agent 1 002, 
as illustrated in Figure 12, then directs two equivalent I/ 
O WRITE requests to data-storage devices 616 and 
628. Similarly, host computer 618 may direct an I/O 
WRITE request to another LUN provided by mirroring 
agent 1002, as shown in Figure 13, where the second 
LUN a virtual LUN with actual data stored on physical 
data-storage devices 623 and 624. As shown in Figure 
14, upon receipt of the I/O WRITE request, the mirroring 
agent 1002 then directs the two equivalent I/O WRITE 
requests to data-storage devices 623 and 624. 
[0038] Although a single mirroring agent may be in- 
corporated into the computing environment illustrated in 



Figure 6, as discussed above with reference to Figures 
1 0-1 4, it is also possible to incorporate multiple mirroring 
agents into the computing environment. Figure 15 
shows the computing environment illustrated in Figure 

5 6 following addition of three mirroring agents 1002, 
1502, and 1504. When multiple mirroring agents are 
available, mirroring initialization and management tasks 
may be shared between them in many different ways. 
For example, mirroring agent 1002 in Figure 15 may in- 

10 itialize and manage the mirroring relationship between 
data-storage devices 61 6 and 628 while mirroring agent 
1504 initializes and manages the mirroring relationship 
between data-storage devices 623 and 624. Mirroring 
agent 1502 may cooperate with mirroring agents 1002 

15 and 1 504 as a fail-over device, to which I/O operations 
and commands can be directed in the case that either 
or both of mirroring agents 1002 and 1504 fail. Alterna- 
tively, a hierarchy of mirroring agents may be estab- 
lished. For example, mirroring agent 1502 may provide 

20 a LUN-based interface to host computers of both com- 
munications medium 602 and 606, and may implement 
the LUN-based interface via LUN-based interfaces pro- 
vided by mirroring agents 1002 and 1504. While mirror- 
ing agents may provide virtual LUNs by storing and ac- 

25 cessing data on remote disk drives and disk arrays, the 
mirroring agent concept is more general, and can en- 
compass initializing and managing mirroring relation- 
ships between a large variety of different types of data 
storage and data manipulation devices. 

30 [0039] Figure 1 6 illustrates the logical components of 
a mirrored data storage device environment provided to 
a host computer via an intermediary hardware-based 
mirroring agent that represents one embodiment of the 
present invention. In Figure 1 6, the mirroring agent 1 602 

35 is attached to a communications medium 1604 along 
with a host computer 1606, a first data storage device 
1608, and a second disk-array data-storage device 
1610. In Figure 16, a single-letter symbolic communica- 
tions-medium address for each of the above-described 

40 devices is shown at the intersection of a coupling line 
between each device and the communications medium 
1604. Thus, data storage device 1608 has communica- 
tions medium address "w," host computer 1606 has 
communications medium address n x, u mirroring agent 

45 1602 has communications medium address "y," and 
disk array 1610 has communications medium address 
"z." The mirroring agent 1 602 includes a controller 1 61 2 
and a memory 1614, similar to equivalent components 
within a disk array. Within the memory, which may be 

so backed up to an internal non-volatile storage compo- 
nent, the mirroring agent controller 1612 maintains a de- 
vice table 1616 and input and output queues 1618 and 
1620. The mirroring agent controller 1612 receives I/O 
requests and commands directed to LUNs provided by 

55 the mirroring agent 1602, queues them in the input 
queue 1618, processes the queued I/O commands and 
operations by retrieving queued operations and com- 
mands from the input queue 1 61 8, and queues outgoing 
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I/O commands and operations directed to data-storage 
devices in output queue 1 620. 
[0040] The host computer 1606 maintains, generally 
within a volume manager, a table 1622 containing infor- 
mation about remote data-storage devices, or volumes, 
accessible to the host computer. Figure 1 6 illustrates im- 
plementation of a mirroring relationship between LUN 1 
provided by data storage device 1 608 and LUN 2 pro- 
vided by disk array 1 61 0 via the mirroring agent 1 602 to 
host computer 1 606. In the internal mirroring agent table 
1616, the mirroring agent controller has stored two en- 
tries representing LUN 2 of disk array 1610 and LUN 1 
1 626 of data storage device 1 608. In each entry, the mir- 
roring agent controller stores a representation of the 
LUN provided by the mirroring agent, in the present case 
LUN M 1, M an indication of the communications medium 
address of a data storage device, in the present case 
"z u in the first entry and M w" in the second entry, an indi- 
cation of the LUN at the physical address, in the present 
case "2" in the first entry and W T in the second entry, 
and additional information about the characteristics and 
parameters of the data-storage devices and configura- 
tion and management parameters associated with the 
data storage devices. By contrast, the host computer ac- 
cesses the virtual LUN provided by the mirroring agent 
1602 directly via the mirroring agents' communications 
medium address, as if it were a physical data storage 
device. Hence, in internal table 1 622 within host com- 
puter 1606, a first entry 1628 represents an accessible 
volume at communications medium address M y," corre- 
sponding to mirroring agent 1602, on LUN "1" provided 
at that address, with various parameters and character- 
istics included in additional columns or fields of the entry, 
and not shown in Figure 16. When the operating system 
of host computer 1606 directs an I/O operation or com- 
mand to the volume associated with LUN "1 M at address 
"y," the I/O command or request is directed to the mir- 
roring agent 1 602, which then processes the I/O request 
or command. If the I/O request is an I/O WRITE request, 
then the controller of the mirroring agent 1612 directs 
two equivalent I/O WRITE requests to data-storage de- 
vices 1608 and 1610, as illustrated in Figure 12. The I/ 
O WRITE requests may be issued together, or one or 
both of the I/O WRITE requests may be deferred to a 
later time. For example, the mirroring agent may queue 
one or both I/O WRITE requests in order to optimize I/ 
O WRITE request processing by the data storage de- 
vices or in order to most efficiently balance servicing of 
incoming requests with processing of already received 
requests. For I/O READ requests, a single I/O READ 
request may be passed by the mirroring agent 1602 to 
either one of the two data-storage devices 1 608 and 
1610. 

[0041] The host computer may also store an indica- 
tion of physical device addresses of data-storage devic- 
es 1608 and 1610 in internal table 1622. Thus, in Figure 
16, the second and third entries of the internal table 
1622 represent physical devices, rather than data stor- 



age volumes, provided as LUNs "1" and "2" at commu- 
nications medium addresses H w n and "z," respectively. 
This allows the host computer to directly access data- 
storage devices 1608 and 1610 via raw I/O requests, 
5 rather than via I/O requests directed through volumes. 
For example, a host computer can direct management 
and administrative commands that do not change the 
data state of the mirrored data-storage devices directly 
to those storage devices, completely bypassing the mir- 

10 roring agent 1 602. Thus, unlike the case of the special- 
ized-hardware mirroring deyice described in the back- 
ground section, a host computer need not direct all I/O 
commands and operations to the mirrored storage de- 
vices via the mirroring agent, but can rationally distribute 

15 I/O commands and operations between the data stor- 
ages devices and the mirroring agent 1602. 
[0042] A hardware-based mirroring agent may re- 
ceive data and status, data, and completion messages 
back from remote data storage devices that the mirror- 

20 ing agent manages, and then forward the status, data, 
and completion messages to host computers that initi- 
ated the I/O requests and commands that generated the 
status, data, and completion messages. In certain im- 
plementations, no status or completion message is re- 

25 turned from mirrored data storage devices upon com- 
pletion of an I/O WRITE request, unless an error occurs. 
[0043] A hardware-based mirroring agent is provided 
the locations and addresses or remote data storage de- 
vices and specifications of mirror relationships to set up 

30 and initialize through a configuration and administration 
interface. The hardware-based mirroring agent then 
provides a LUN-based interface to the remote data stor- 
age devices via a communications medium to host com- 
puters. A host computer can be directed to remap re- 

35 mote devices accessible via the communications medi- 
um via an automated discovery process, during which 
volume manager tables or host I/O tables are updated 
to reflect locations, addresses, and parameters and 
characteristics of devices detected by the host compu- 

40 ter. The mirroring agent establishes and synchronizes 
groups of mirrored data storage devices using well- 
known disk mirroring techniques. 
[0044] As with any mirrored data storage devices 
manager, a mirroring agent is responsible for detecting 

45 failures of mirrored devices, appropriated failing over to 
surviving data storage devices within a mirror group, 
and synchronizing failed devices that are repaired and 
brought back to operating status. These tasks are well- 
known within the art, and will not be further described. 

50 The mirroring agent also manages concurrent updating 
of ail data storage devices within a group of mirrored 
data storage devices to provide for immediate fail-over 
when one or more data storage devices within a group 
of mirrored data storage devices fail. 

55 [0045] The mirroring agent is not a single point of fail- 
ure with respect to data access, since the mirroring 
agent can include redundant components and since 
host computers may access data storage devices as 
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physical devices to bypass a failed mirroring agent. 
Configuration changes of mirrored data storage devices 
can be undertaken by a mirroring agent transparently 
with respect to host computers, without disrupting or 
even interrupting I/O activity. By correctly routing I/O 
WRITE requests and other I/O requests and commands 
that change the data state of data storage devices to a 
mirroring agent, and routing all other I/O requests and 
commands directly to data storage devices, the poten- 
tial bottleneck otherwise represented by mirroring 
agents can be avoided. Hosts of any type and configu- 
ration may access a mirroring agent through well-known 
interfaces provided by disk arrays, minimizing host com- 
puter support required by mirroring agents. 
[0046] Although the present invention has been de- 
scribed in terms of a particular embodiment, it is not in- 
tended that the invention be limited to this embodiment 
Modifications within the spirit of the invention will be ap- 
parent to those skilled in the art. For example, the hard- 
ware-based mirroring agent can be extended concep- 
tually to provide mirroring, and other administrative and 
configurational tasks, to remotely accessible devices of 
many different types, in addition to data storage devices. 
As discussed above, hardware-based mirroring agents 
may cooperate to provide more robust and more failure 
tolerant mirroring capabilities, and hardware-based mir- 
roring agents can be interconnected in more complex 
topologies, including hierarchies and graphs, to further 
these ends. As discussed above, the hardware-based 
mirroring agent that represents one embodiment of the 
present invention contains many of the same compo- 
nents as, and is internally organized similar to, a disk 
array. Just like a disk array, the hardware-based mirror- 
ing agent receives I/O commands and requests via a 
communications medium and processes those re- 
quests, using request-processing routines that run on a 
controller and using various information stored and 
maintained in a memory component. The hardware- 
based mirroring agent, like disk arrays, may also include 
interconnection ports and many other components. 
However, unlike disk arrays, the hardware-based mir- 
roring agent manages, and provides an interface to, re- 
mote data storage devices and other remote devices 
rather than to internal data storage devices, as in the 
case of a disk array. As with disk arrays, there are an 
almost limitless number of different hardware, firmware, 
and software implementations of the various internal 
components. Implementation of I/O request and com- 
mand processing is dependent on the communications 
medium through which the hardware-based mirroring 
agent accesses the data storage devices which it pro- 
vides an interface to, and the communications protocols 
through which the hardware-based mirroring agent in- 
tercommunicates with host computers is also depend- 
ent on the communications medium and on the protocol 
engines employed by the host computers. Many of 
these different alternative embodiments of hardware, 
firmware, and software components are well-known to 



designers and manufacturers of disk arrays. A hard- 
ware-based mirroring agent may be interconnected to 
various remote devices through one or more communi- 
cations media of various different types, and can inter- 

5 communicate with one or more host computers. The 
hardware-based mirroring agent can provide well- 
known and common mirroring functionality including 
mirror splits, dynamic reconfiguration of mirrors, dynam- 
ic addition of additional physical devices to a group of 

10 mirrored devices, mirror concurrency and, synchroniza- 
tion, and mirror initialization. The hardware-based mir- 
roring agent may combine two or more physical devices 
to create a logical mirrored device. In addition to mirror- 
ing, a hardware-based mirroring agent may also under- 

15 take various backup, archival, data compression and 
decompression, and other data manipulation operations 
for virtual LUNs provided to host computers implement- 
ed via remote data storage systems. The described em- 
bodiment featured a fibre channel communications me- 

20 dium, but any other communication medium that can 
support data storage and retrieval operations between 
host computers and data storage devices can also be 
employed to interconnect the mirroring agent with host 
computers and data storage devices, including commu- 

25 nications media supporting the Internet Protocol, and 
others. Additionally, multiple communications media 
may be used for host/mirroring agent interconnection 
and mirroring agent/data storage device interconnec- 
tion. 

30 [0047] The foregoing description, for purposes of ex- 
planation, used specific nomenclature to provide a thor- 
ough understanding of the invention. However, it will be 
apparent to one skilled in the art that the specific details 
are not required in order to practice the invention. The 

35 foregoing descriptions of specific embodiments of the 
present invention are presented for purpose of illustra- 
tion and description. They are not intended to be ex- 
haustive or to limit the invention to the precise forms dis- 
closed. Obviously many modifications and variations 

40 are possible in view of the above teachings. The em- 
bodiments are shown and described in order to best ex- 
plain the principles of the invention and its practical ap- 
plications, to thereby enable others skilled in the art to 
best utilize the invention and various embodiments with 

45 various modifications as are suited to the particular use 
contemplated. It is intended that the scope of the inven- 
tion be defined by the following claims and their equiv- 
alents: 

50 

Claims 

1 . A mirroring agent (1 602) comprising: 

55 a communications port to a communications . 

medium (1604) through which the mirroring 
agent receives requests, commands, and sta- 
tus messages and through which the mirroring 



9 



17 



EP 1 229 447 A2 



18 



agent sends requests and commands, the 
communications port associated with an ad- 
dress; 

a memory (1614) that stores received incoming 
requests and commands and outgoing re- 5 
quests and commands received and transmit- 
ted via the communications port; and 
a controller (1612) that executes routines that 
provide a virtual logical unit interface to a host 
computer (1 606) accessing the mirroring agent io 
via the communications medium and that im- 
plement the virtual logical unit interface by is- 
suing requests and commands to at least two 
data storage devices (1608, 1610) remote to 
the mirroring agent and accessible by the mir- is 
roring agent via the communications medium 
(1604) and communications medium address- 
es associated with the at least two data storage 
devices, the at least two data storage devices 
continuously updated to be mirrors of one an- 20 
other. 

2. The mirroring agent (1602) of claim 1 wherein 

the communications medium (1604) is a fibre 
channel communications medium; 25 

the communications medium supports the In- 
ternet Protocol; 

the at least two data storage devices (1608, 
1610) are disk arrays. 

30 

3. The mirroring agent (1602) of claim 1 wherein the 
host computer (1606) discovers the virtual logical 
unit, via a communications medium discovery proc- 
ess, at the communications address associated 
with the mirroring agent, and maintains an internal 35 
indication (1628) associating the virtual logical unit 
with an indication of the communications address 
associated with the mirroring agent. 

4. The mirroring agent of claim 3 *o 

wherein the host computer (1 606) discovers 
the at least two data storage devices(1608, 1610), 
via a communications medium discovery process, 
at the communications address associated with the 
at least two data storage devices, and maintains in- 45 
ternal indications (1 622) associating the at least two 
data storage devices with indications of the commu- 
nications addresses associated with the at least two 
data storage devices; and 

wherein the host computer (1606) directs so 
read and write requests to the virtual logical unit via 
the communications address associated with the 
mirroring agent (1602), and may direct administra- 
tive and management commands directly to one or 
more of the at least two data storage devices (1 608, 55 
1610) via one or more of the communications ad- 
dresses associated with the at least two data stor- 
age devices. 



5. A method for providing mirrored data storage devic- 
es to a host computer (1606), the method compris- 
ing: 

providing a mirroring agent (1 602) that includes 
a communications medium port, memory 
(1614), and a controller (1612), and that is as- 
sociated with a communications medium ad- 
dress; 

providing to the host computer (1606) via a 
communications medium (1604) a logical unit 
interface; and 

implementing the logical unit interface by the 
mirroring agent (1602) by 

receiving requests and commands from 
the host computer directed to the logical unit 
interface, and 

forwarding requests and commands to at 
least two data storage devices (1 608, 1 61 0) re- 
mote from the mirroring agent and associated 
with communications medium addresses via 
the communications medium so that the at least 
two data storage devices are continuously up- 
dated to each maintain a single consistent data 
state that corresponds to a mirror relationship 
among the at least two data storage devices. 

6. The method of claim 5 further including receiving, 
by the mirroring agent (1 602), directives to establish 
a mirror relationship between at least two data stor- 
age devices (1 608, 1610), whereupon the mirroring 
agent synchronizes the at least two data storage 
devices and provides a logical unit interface to the 
at least two data storage devices. 

7. The method of claim 5 further including discovering, 
by the host computer (1606), the logical unit inter- 
face provided by the mirroring agent (1602) to the 
at least two data storage devices (1608, 1610) via 
a communications medium discovery process, and 
storing an indication of the communications ad- 
dress associated with the mirroring device along 
with an indication (1628) of the logical unit interface. 

8. The method of claim 5 further including discovering, 
by the host computer (1606), the communications 
addresses associated with the at least two data 
storage devices (1 608, 1 61 0) via a communications 
medium discovery process, and storing indications 
of the communications addresses associated within 
the at least two data storage devices (1 608, 1610). 

9. The method of claim 8 further including accessing 
the at least two data storage devices (1608, 1610) 
by the* host computer (1606) via the logical unit in- 
terface provided by the mirroring agent (1 602) using 
the stored indication (1622) of the communications 
address associated with the mirroring device 
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(1602). 

10. The method of claim 9 further including accessing 
the at least two data storage devices (1608, 1610) 
directly by the host computer (1606) using the 5 
stored indications of the communications address- 
es associated with the at least two data storage de- 
vices (1608, 1610). 
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PROGRAM 

fopen ("/dev/source/prog.cc", V); 
fread (buf, 256, 35, openstr); 



FILE MANAGER 

/dev/source/prog.cc 
/usr/CCB/bin/prog 



VOLUME 



VOLUME MANAGER 
vol[0-n] 



-408 



-OBJECT NAME 
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-404 



LUN 



DISK ARRAY 
disk/track/sector/byte 
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